Configurable histogram-of-oriented gradients (HOG) processor

ABSTRACT

Embodiments relate to a histogram-of-oriented gradients (HOG) module. The HOG module is implemented in hardware rather than software. The HOG module applies an algorithm to an image to identify gradient orientation in localized portions of the image. The HOG module creates a histogram-of orientation gradients based on the identified gradient orientations.

BACKGROUND

Image data captured by an image sensor or received from other datasources is often processed in an image processing pipeline beforefurther processing or consumption. For example, raw image data may becorrected, filtered, or otherwise modified before being provided tosubsequent components such as a video encoder. To perform corrections orenhancements for captured image data, various components, unit stages ormodules may be employed.

Such an image processing pipeline may be structured so that correctionsor enhancements to the captured image data can be performed in anexpedient way. For example, the image processing pipeline may employ animage processing algorithm used to calculate histograms-of-orientedgradients (HOG) of image data. The image processing algorithm istypically performed by executing software programs on a centralprocessing unit (CPU). However, execution of such programs on the CPUconsumes significant bandwidth of the CPU and other peripheral resourcesas well as increase power consumption.

SUMMARY

The embodiments herein describe a histogram-of-oriented gradients (HOG)module that is implemented in hardware rather than software. The HOGmodule includes configuration registers that store configuration valuesthat set operation parameters of the HOG module. The HOG modulegenerates histograms of different types of HOG features of image databased on configuration values stored in the configuration registers.

The embodiments herein also describe a patch mode of the HOG module.During the patch mode, the HOG module processes image data that includesmultiple image patches. The image patches may be extracted fromdifferent source images or from a single source image. The HOG moduleprocesses the image data to generate a histogram of gradient featuresfor each image patch included in the image.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a high-level diagram of an electronic device, according to oneembodiment

FIG. 2 is a block diagram illustrating components in the electronicdevice, according to one embodiment.

FIG. 3 is a block diagram illustrating image processing pipelinesimplemented using an image signal processor, according to oneembodiment.

FIG. 4 is a block diagram illustrating a histogram-of-oriented gradients(HOG) module of the image signal processor, according to one embodiment.

FIG. 5 is a block diagram illustrating a detailed view of a HOG enginecircuit included in the HOG module, according to one embodiment.

FIG. 6 is a diagram illustrating various types of HOG data outputted bythe HOG module, according to one embodiment.

FIG. 7 is a diagram illustrating an image including image patches forprocessing by the HOG module, according to one embodiment.

FIGS. 8A, 8B, 8C, and 8D are diagrams illustrating different images thatinclude the image patches shown in FIG. 7, according to one embodiment.

FIG. 9 is a flowchart illustrating a method of the HOG module forprocessing an image, according to one embodiment.

FIG. 10 is a flowchart illustrating a method of the HOG module forperforming patch processing of an image, according to one embodiment.

The figures depict, and the detail description describes, variousnon-limiting embodiments for purposes of illustration only.

DETAILED DESCRIPTION

Reference will now be made in detail to embodiments, examples of whichare illustrated in the accompanying drawings. In the following detaileddescription, numerous specific details are set forth in order to providea thorough understanding of the various described embodiments. However,the described embodiments may be practiced without these specificdetails. In other instances, well-known methods, procedures, components,circuits, and networks have not been described in detail so as not tounnecessarily obscure aspects of the embodiments.

Embodiments of the present disclosure relate to a histogram-of-orientedgradients (HOG) module that is implemented in hardware rather thansoftware. The HOG module applies an algorithm to an image to identifygradient orientation in localized portions of the image. The HOG modulegenerates gradient features of the image such as a histogram-of-orientedgradients based on the identified gradient orientations. By implementingthe HOG module in hardware rather than software, the HOG module reducesbandwidth and power consumption of the CPU that is typically responsiblefor creating gradient features of images.

Furthermore, the HOG module can operate in a patch mode. During thepatch mode, the HOG module processes an image composed of multiple imagepatches. The image patches may be extracted from different source imagesor from a single source image. The HOG module can identify gradientfeatures for each of the image patches included in the image. Byoperating in the patch mode, the HOG module can efficiently process thedifferent image patches in a single image rather than process thedifferent images from which the different image patches were extractedor process the entire source image from which the image patches wereextracted.

Exemplary Electronic Device

Embodiments of electronic devices, user interfaces for such devices, andassociated processes for using such devices are described. In someembodiments, the device is a portable communications device, such as amobile telephone, that also contains other functions, such as personaldigital assistant (PDA) and/or music player functions. Exemplaryembodiments of portable multifunction devices include, withoutlimitation, the iPhone®, iPod Touch®, Apple Watch®, and iPad® devicesfrom Apple Inc. of Cupertino, Calif. Other portable electronic devices,such as wearables, laptops or tablet computers, are optionally used. Insome embodiments, the device is not a portable communications device,but is a desktop computer or other computing device that is not designedfor portable use. In some embodiments, the disclosed electronic devicemay include a touch sensitive surface (e.g., a touch screen displayand/or a touch pad). An example electronic device described below inconjunction with FIG. 1 (e.g., device 100) may include a touch-sensitivesurface for receiving user input. The electronic device may also includeone or more other physical user-interface devices, such as a physicalkeyboard, a mouse and/or a joystick.

FIG. 1 is a high-level diagram of an electronic device 100, according toone embodiment. Device 100 may include one or more physical buttons,such as a “home” or menu button 104. Menu button 104 is, for example,used to navigate to any application in a set of applications that areexecuted on device 100. In some embodiments, menu button 104 includes afingerprint sensor that identifies a fingerprint on menu button 104. Thefingerprint sensor may be used to determine whether a finger on menubutton 104 has a fingerprint that matches a fingerprint stored forunlocking device 100. Alternatively, in some embodiments, menu button104 is implemented as a soft key in a graphical user interface (GUI)displayed on a touch screen.

In some embodiments, device 100 includes touch screen 150, menu button104, push button 106 for powering the device on/off and locking thedevice, volume adjustment buttons 108, Subscriber Identity Module (SIM)card slot 110, head set jack 112, and docking/charging external port124. Push button 106 may be used to turn the power on/off on the deviceby depressing the button and holding the button in the depressed statefor a predefined time interval; to lock the device by depressing thebutton and releasing the button before the predefined time interval haselapsed; and/or to unlock the device or initiate an unlock process. Inan alternative embodiment, device 100 also accepts verbal input foractivation or deactivation of some functions through microphone 113. Thedevice 100 includes various components including, but not limited to, amemory (which may include one or more computer readable storagemediums), a memory controller, one or more central processing units(CPUs), a peripherals interface, an RF circuitry, an audio circuitry,speaker 111, microphone 113, input/output (I/O) subsystem, and otherinput or control devices. Device 100 may include one or more imagesensors 164, one or more proximity sensors 166, and one or moreaccelerometers 168. The device 100 may include components not shown inFIG. 1.

Device 100 is only one example of an electronic device, and device 100may have more or fewer components than listed above, some of which maybe combined into a components or have a different configuration orarrangement. The various components of device 100 listed above areembodied in hardware, software, firmware or a combination thereof,including one or more signal processing and/or application specificintegrated circuits (ASICs).

FIG. 2 is a block diagram illustrating components in device 100,according to one embodiment. Device 100 may perform various operationsincluding image processing. For this and other purposes, the device 100may include, among other components, image sensor 202, system-on-a chip(SOC) component 204, system memory 230, persistent storage (e.g., flashmemory) 228, orientation sensor 234, and display 216. The components asillustrated in FIG. 2 are merely illustrative. For example, device 100may include other components (such as speaker or microphone) that arenot illustrated in FIG. 2. Further, some components (such as orientationsensor 234) may be omitted from device 100.

Image sensor 202 is a component for capturing image data and may beembodied, for example, as a complementary metal-oxide-semiconductor(CMOS) active-pixel sensor a camera, video camera, or other devices.Image sensor 202 generates raw image data that is sent to SOC component204 for further processing. In some embodiments, the image dataprocessed by SOC component 204 is displayed on display 216, stored insystem memory 230, persistent storage 228 or sent to a remote computingdevice via network connection. The raw image data generated by imagesensor 202 may be in a Bayer color filter array (CFA) pattern(hereinafter also referred to as “Bayer pattern”).

Motion sensor 234 is a component or a set of components for sensingmotion of device 100. Motion sensor 234 may generate sensor signalsindicative of orientation and/or acceleration of device 100. The sensorsignals are sent to SOC component 204 for various operations such asturning on device 100 or rotating images displayed on display 216.

Display 216 is a component for displaying images as generated by SOCcomponent 204. Display 216 may include, for example, liquid crystaldisplay (LCD) device or an organic light emitting diode (OLED) device.Based on data received from SOC component 204, display 116 may displayvarious images, such as menus, selected operating parameters, imagescaptured by image sensor 202 and processed by SOC component 204, and/orother information received from a user interface of device 100 (notshown).

System memory 230 is a component for storing instructions for executionby SOC component 204 and for storing data processed by SOC component204. System memory 230 may be embodied as any type of memory including,for example, dynamic random access memory (DRAM), synchronous DRAM(SDRAM), double data rate (DDR, DDR2, DDR3, etc.) RAMBUS DRAM (RDRAM),static RAM (SRAM) or a combination thereof. In some embodiments, systemmemory 230 may store pixel data or other image data or statistics invarious formats.

Persistent storage 228 is a component for storing data in a non-volatilemanner. Persistent storage 228 retains data even when power is notavailable. Persistent storage 228 may be embodied as read-only memory(ROM), flash memory or other non-volatile random access memory devices.

SOC component 204 is embodied as one or more integrated circuit (IC)chip and performs various data processing processes. SOC component 204may include, among other subcomponents, image signal processor (ISP)206, a central processor unit (CPU) 208, a network interface 210, sensorinterface 212, display controller 214, graphics processor (GPU) 220,memory controller 222, video encoder 224, storage controller 226, andvarious other input/output (I/O) interfaces 218, and bus 232 connectingthese subcomponents. SOC component 204 may include more or fewersubcomponents than those shown in FIG. 2.

ISP 206 is hardware that performs various stages of an image processingpipeline. In some embodiments, ISP 206 may receive raw image data fromimage sensor 202, and process the raw image data into a form that isusable by other subcomponents of SOC component 204 or components ofdevice 100. ISP 206 may perform various image-manipulation operationssuch as image translation operations, horizontal and vertical scaling,color space conversion and/or image stabilization transformations, asdescribed below in detail with reference to FIG. 3.

CPU 208 may be embodied using any suitable instruction set architecture,and may be configured to execute instructions defined in thatinstruction set architecture. CPU 208 may be general-purpose or embeddedprocessors using any of a variety of instruction set architectures(ISAs), such as the x86, PowerPC, SPARC, RISC, ARM or MIPS ISAs, or anyother suitable ISA. Although a single CPU is illustrated in FIG. 2, SOCcomponent 204 may include multiple CPUs. In multiprocessor systems, eachof the CPUs may commonly, but not necessarily, implement the same ISA.

Graphics processing unit (GPU) 220 is graphics processing circuitry forperforming graphical data. For example, GPU 220 may render objects to bedisplayed into a frame buffer (e.g., one that includes pixel data for anentire frame). GPU 220 may include one or more graphics processors thatmay execute graphics software to perform a part or all of the graphicsoperation, or hardware acceleration of certain graphics operations.

I/O interfaces 218 are hardware, software, firmware or combinationsthereof for interfacing with various input/output components in device100. I/O components may include devices such as keypads, buttons, audiodevices, and sensors such as a global positioning system. I/O interfaces218 process data for sending data to such I/O components or process datareceived from such I/O components.

Network interface 210 is a subcomponent that enables data to beexchanged between devices 100 and other devices via one or more networks(e.g., carrier or agent devices). For example, video or other image datamay be received from other devices via network interface 210 and bestored in system memory 230 for subsequent processing (e.g., via aback-end interface to image signal processor 206, such as discussedbelow in FIG. 3) and display. The networks may include, but are notlimited to, Local Area Networks (LANs) (e.g., an Ethernet or corporatenetwork) and Wide Area Networks (WANs). The image data received vianetwork interface 210 may undergo image processing processes by ISP 206.

Sensor interface 212 is circuitry for interfacing with motion sensor234. Sensor interface 212 receives sensor information from motion sensor234 and processes the sensor information to determine the orientation ormovement of the device 100.

Display controller 214 is circuitry for sending image data to bedisplayed on display 216. Display controller 214 receives the image datafrom ISP 206, CPU 208, graphic processor or system memory 230 andprocesses the image data into a format suitable for display on display216.

Memory controller 222 is circuitry for communicating with system memory230. Memory controller 222 may read data from system memory 230 forprocessing by ISP 206, CPU 208, GPU 220 or other subcomponents of SOCcomponent 204. Memory controller 222 may also write data to systemmemory 230 received from various subcomponents of SOC component 204.

Video encoder 224 is hardware, software, firmware or a combinationthereof for encoding video data into a format suitable for storing inpersistent storage 128 or for passing the data to network interface 210for transmission over a network to another device.

In some embodiments, one or more subcomponents of SOC component 204 orsome functionality of these subcomponents may be performed by softwarecomponents executed on ISP 206, CPU 208 or GPU 220. Such softwarecomponents may be stored in system memory 230, persistent storage 228 oranother device communicating with device 100 via network interface 210.

Image data or video data may flow through various data paths within SOCcomponent 204. In one example, raw image data may be generated from theimage sensor 202 and processed by ISP 206, and then sent to systemmemory 230 via bus 232 and memory controller 222. After the image datais stored in system memory 230, it may be accessed by video encoder 224for encoding or by display 116 for displaying via bus 232.

In another example, image data is received from sources other than theimage sensor 202. For example, video data may be streamed, downloaded,or otherwise communicated to the SOC component 204 via wired or wirelessnetwork. The image data may be received via network interface 210 andwritten to system memory 230 via memory controller 222. The image datamay then be obtained by ISP 206 from system memory 230 and processedthrough one or more image processing pipeline stages, as described belowin detail with reference to FIG. 3. The image data may then be returnedto system memory 230 or be sent to video encoder 224, display controller214 (for display on display 216), or storage controller 226 for storageat persistent storage 228.

Example Image Signal Processing Pipelines

FIG. 3 is a block diagram illustrating image processing pipelinesimplemented using ISP 206, according to one embodiment. In theembodiment of FIG. 3, ISP 206 is coupled to image sensor 202 to receiveraw image data. ISP 206 implements an image processing pipeline whichmay include a set of stages that process image information fromcreation, capture or receipt to output. ISP 206 may include, among othercomponents, sensor interface 302, central control 320, front-endpipeline stages 330, back-end pipeline stages 340, image statisticsmodule 304, vision module 322, back-end interface 342, and outputinterface 316. ISP 206 may include other components not illustrated inFIG. 3 or may omit one or more components illustrated in FIG. 3.

In one or more embodiments, different components of ISP 206 processimage data at different rates. In the embodiment of FIG. 3, front-endpipeline stages 330 (e.g., raw processing stage 306 and resampleprocessing stage 308) may process image data at an initial rate. Thus,the various different techniques, adjustments, modifications, or otherprocessing operations performed by these front-end pipeline stages 330at the initial rate. For example, if the front-end pipeline stages 330process 2 pixels per clock cycle, then raw processing stage 308operations (e.g., black level compensation, highlight recovery anddefective pixel correction) may process 2 pixels of image data at atime. In contrast, one or more back-end pipeline stages 340 may processimage data at a different rate less than the initial data rate. Forexample, in the embodiment of FIG. 3, back-end pipeline stages 340(e.g., noise processing stage 310, color processing stage 312, andoutput rescale 314) may be processed at a reduced rate (e.g., 1 pixelper clock cycle).

Sensor interface 302 receives raw image data from image sensor 202 andprocesses the raw image data into an image data processable by otherstages in the pipeline. Sensor interface 302 may perform variouspreprocessing operations, such as image cropping, binning or scaling toreduce image data size. In some embodiments, pixels are sent from theimage sensor 202 to sensor interface 302 in raster order (i.e.,horizontally, line by line). The subsequent processes in the pipelinemay also be performed in raster order and the result may also be outputin raster order. Although only a single image sensor and a single sensorinterface 302 are illustrated in FIG. 3, when more than one image sensoris provided in device 100, a corresponding number of sensor interfacesmay be provided in ISP 206 to process raw image data from each imagesensor.

Front-end pipeline stages 330 process image data in raw or full-colordomains. Front-end pipeline stages 330 may include, but are not limitedto, raw processing stage 306 and resample processing stage 308. A rawimage data may be in Bayer raw format, for example. In Bayer raw imageformat, pixel data with values specific to a particular color (insteadof all colors) is provided in each pixel. In an image capturing sensor,image data is typically provided in a Bayer pattern. Raw processingstage 308 may process image data in a Bayer raw format.

The operations performed by raw processing stage 308 include, but arenot limited, sensor linearization, black level compensation, fixedpattern noise reduction, defective pixel correction, raw noisefiltering, lens shading correction, white balance gain, and highlightrecovery. Sensor linearization refers to mapping non-linear image datato linear space for other processing. Black level compensation refers toproviding digital gain, offset and clip independently for each colorcomponent (e.g., Gr, R, B, Gb) of the image data. Fixed pattern noisereduction refers to removing offset fixed pattern noise and gain fixedpattern noise by subtracting a dark frame from an input image andmultiplying different gains to pixels. Defective pixel correction refersto detecting defective pixels, and then replacing defective pixelvalues. Raw noise filtering refers to reducing noise of image data byaveraging neighbor pixels that are similar in brightness. Highlightrecovery refers to estimating pixel values for those pixels that areclipped (or nearly clipped) from other channels. Lens shading correctionrefers to applying a gain per pixel to compensate for a dropoff inintensity roughly proportional to a distance from a lens optical center.White balance gain refers to providing digital gains for white balance,offset and clip independently for all color components (e.g., Gr, R, B,Gb in Bayer format). Components of ISP 206 may convert raw image datainto image data in full-color domain, and thus, raw processing stage 308may process image data in the full-color domain in addition to orinstead of raw image data.

Resample processing stage 308 performs various operations to convert,resample, or scale image data received from raw processing stage 306.Operations performed by resample processing stage 308 may include, butnot limited to, demosaic operation, per-pixel color correctionoperation, Gamma mapping operation, color space conversion anddownscaling or sub-band splitting. Demosaic operation refers toconverting or interpolating missing color samples from raw image data(for example, in a Bayer pattern) to output image data into a full-colordomain. Demosaic operation may include low pass directional filtering onthe interpolated samples to obtain full-color pixels. Per-pixel colorcorrection operation refers to a process of performing color correctionon a per-pixel basis using information about relative noise standarddeviations of each color channel to correct color without amplifyingnoise in the image data. Gamma mapping refers to converting image datafrom input image data values to output data values to perform specialimage effects, including black and white conversion, sepia toneconversion, negative conversion, or solarize conversion. For the purposeof Gamma mapping, lookup tables (or other structures that index pixelvalues to another value) for different color components or channels ofeach pixel (e.g., a separate lookup table for Y, Cb, and Cr colorcomponents) may be used. Color space conversion refers to convertingcolor space of an input image data into a different format. In oneembodiment, resample processing stage 308 converts RBD format into YCbCrformat for further processing.

Central control module 320 may control and coordinate overall operationof other components in ISP 206. Central control module 320 performsoperations including, but not limited to, monitoring various operatingparameters (e.g., logging clock cycles, memory latency, quality ofservice, and state information), updating or managing control parametersfor other components of ISP 206, and interfacing with sensor interface302 to control the starting and stopping of other components of ISP 206.For example, central control module 320 may update programmableparameters for other components in ISP 206 while the other componentsare in an idle state. After updating the programmable parameters,central control module 320 may place these components of ISP 206 into arun state to perform one or more operations or tasks. Central controlmodule 320 may also instruct other components of ISP 206 to store imagedata (e.g., by writing to system memory 230 in FIG. 2) before, during,or after resample processing stage 308. In this way full-resolutionimage data in raw or full-color domain format may be stored in additionto or instead of processing the image data output from resampleprocessing stage 308 through backend pipeline stages 340.

Image statistics module 304 performs various operations to collectstatistic information associated with the image data. The operations forcollecting statistics information may include, but not limited to,sensor linearization, mask patterned defective pixels, sub-sample rawimage data, detect and replace non-patterned defective pixels, blacklevel compensation, lens shading correction, and inverse black levelcompensation. After performing one or more of such operations,statistics information such as 3A statistics (Auto white balance (AWB),auto exposure (AE), auto focus (AF)), histograms (e.g., 2D color orcomponent) and any other image data information may be collected ortracked. In some embodiments, certain pixels' values, or areas of pixelvalues may be excluded from collections of certain statistics data(e.g., AF statistics) when preceding operations identify clipped pixels.Although only a single statistics module 304 is illustrated in FIG. 3,multiple image statistics modules may be included in ISP 206. In suchembodiments, each statistic module may be programmed by central controlmodule 320 to collect different information for the same or differentimage data.

Vision module 322 performs various operations to facilitate computervision operations at CPU 208 such as facial detection in image data. Thevision module 322 may perform various operations includingpre-processing, global tone-mapping and Gamma correction, vision noisefiltering, resizing, keypoint detection, convolution and generation ofhistogram-of-oriented gradients (HOG) data. The pre-processing mayinclude subsampling or binning operation and computation of luminance ifthe input image data is not in YCrCb format. Global mapping and Gammacorrection can be performed on the pre-processed data on luminanceimage. Vision noise filtering is performed to remove pixel defects andreduce noise present in the image data, and thereby, improve the qualityand performance of subsequent computer vision algorithms. Such visionnoise filtering may include detecting and fixing dots or defectivepixels, and performing bilateral filtering to reduce noise by averagingneighbor pixels of similar brightness. Various vision algorithms useimages of different sizes and scales. Resizing of an image is performed,for example, by binning or linear interpolation operation. Keypoints arelocations within an image that are surrounded by image patches wellsuited to matching in other images of the same scene or object. Suchkeypoints are useful in image alignment, computing cameral pose andobject tracking. Keypoint detection refers to the process of identifyingsuch keypoints in an image. Convolution is heavily used tools inimage/video processing and machine vision. Convolution may be performed,for example, to generate edge maps of images or smoothen images. HOGprovides descriptions of image patches for tasks in image analysis andcomputer vision. HOG can be generated, for example, by (i) computinghorizontal and vertical gradients using a simple difference filter, (ii)computing gradient orientations and magnitudes from the horizontal andvertical gradients, and (iii) binning the gradient orientations.

Back-end interface 342 receives image data from other image sources thanimage sensor 202 and forwards it to other components of ISP 206 forprocessing. For example, image data may be received over a networkconnection and be stored in system memory 230. Back-end interface 342retrieves the image data stored in system memory 230 and provide it toback-end pipeline stages 340 for processing. One of many operations thatare performed by back-end interface 342 is converting the retrievedimage data to a format that can be utilized by back-end processingstages 340. For instance, back-end interface 342 may convert RGB, YCbCr4:2:0, or YCbCr 4:2:2 formatted image data into YCbCr 4:4:4 colorformat.

Back-end pipeline stages 340 processes image data according to aparticular full-color format (e.g., YCbCr 4:4:4 or RGB). In someembodiments, components of the back-end pipeline stages 340 may convertimage data to a particular full-color format before further processing.Back-end pipeline stages 340 may include, among other stages, noiseprocessing stage 310 and color processing stage 312. Back-end pipelinestages 340 may include other stages not illustrated in FIG. 3.

Noise processing stage 310 performs various operations to reduce noisein the image data. The operations performed by noise processing stage310 include, but are not limited to, color space conversion,gamma/de-gamma mapping, temporal filtering, noise filtering, lumasharpening, and chroma noise reduction. The color space conversion mayconvert an image data from one color space format to another color spaceformat (e.g., RGB format converted to YCbCr format). Gamma/de-gammaoperation converts image data from input image data values to outputdata values to perform special image effects. Temporal filtering filtersnoise using a previously filtered image frame to reduce noise. Forexample, pixel values of a prior image frame are combined with pixelvalues of a current image frame. Noise filtering may include, forexample, spatial noise filtering. Luma sharpening may sharpen lumavalues of pixel data while chroma suppression may attenuate chroma togray (i.e. no color). In some embodiment, the luma sharpening and chromasuppression may be performed simultaneously with spatial nose filtering.The aggressiveness of noise filtering may be determined differently fordifferent regions of an image. Spatial noise filtering may be includedas part of a temporal loop implementing temporal filtering. For example,a previous image frame may be processed by a temporal filter and aspatial noise filter before being stored as a reference frame for a nextimage frame to be processed. In other embodiments, spatial noisefiltering may not be included as part of the temporal loop for temporalfiltering (e.g., the spatial noise filter may be applied to an imageframe after it is stored as a reference image frame (and thus is not aspatially filtered reference frame).

Color processing stage 312 may perform various operations associatedwith adjusting color information in the image data. The operationsperformed in color processing stage 312 include, but are not limited to,local tone mapping, gain/offset/clip, color correction,three-dimensional color lookup, gamma conversion, and color spaceconversion. Local tone mapping refers to spatially varying local tonecurves in order to provide more control when rendering an image. Forinstance, a two-dimensional grid of tone curves (which may be programmedby the central control module 320) may be bi-linearly interpolated suchthat smoothly varying tone curves are created across an image. In someembodiments, local tone mapping may also apply spatially varying andintensity varying color correction matrices, which may, for example, beused to make skies bluer while turning down blue in the shadows in animage. Digital gain/offset/clip may be provided for each color channelor component of image data. Color correction may apply a colorcorrection transform matrix to image data. 3D color lookup may utilize athree dimensional array of color component output values (e.g., R, G, B)to perform advanced tone mapping, color space conversions, and othercolor transforms. Gamma conversion may be performed, for example, bymapping input image data values to output data values in order toperform gamma correction, tone mapping, or histogram matching. Colorspace conversion may be implemented to convert image data from one colorspace to another (e.g., RGB to YCbCr). Other processing techniques mayalso be performed as part of color processing stage 312 to perform otherspecial image effects, including black and white conversion, sepia toneconversion, negative conversion, or solarize conversion.

Output rescale module 314 may resample, transform and correct distortionon the fly as the ISP 206 processes image data. Output rescale module314 may compute a fractional input coordinate for each pixel and usesthis fractional coordinate to interpolate an output pixel via apolyphase resampling filter. A fractional input coordinate may beproduced from a variety of possible transforms of an output coordinate,such as resizing or cropping an image (e.g., via a simple horizontal andvertical scaling transform), rotating and shearing an image (e.g., vianon-separable matrix transforms), perspective warping (e.g., via anadditional depth transform) and per-pixel perspective divides applied inpiecewise in strips to account for changes in image sensor during imagedata capture (e.g., due to a rolling shutter), and geometric distortioncorrection (e.g., via computing a radial distance from the opticalcenter in order to index an interpolated radial gain table, and applyinga radial perturbance to a coordinate to account for a radial lensdistortion).

Output rescale module 314 may apply transforms to image data as it isprocessed at output rescale module 314. Output rescale module 314 mayinclude horizontal and vertical scaling components. The vertical portionof the design may implement series of image data line buffers to holdthe “support” needed by the vertical filter. As ISP 206 may be astreaming device, it may be that only the lines of image data in afinite-length sliding window of lines are available for the filter touse. Once a line has been discarded to make room for a new incomingline, the line may be unavailable. Output rescale module 314 maystatistically monitor computed input Y coordinates over previous linesand use it to compute an optimal set of lines to hold in the verticalsupport window. For each subsequent line, output rescale module mayautomatically generate a guess as to the center of the vertical supportwindow. In some embodiments, output rescale module 314 may implement atable of piecewise perspective transforms encoded as digital differenceanalyzer (DDA) steppers to perform a per-pixel perspectivetransformation between a input image data and output image data in orderto correct artifacts and motion caused by sensor motion during thecapture of the image frame. Output rescale may provide image data viaoutput interface 314 to various other components of system 100, asdiscussed above with regard to FIGS. 1 and 2.

In various embodiments, the functionally of components 302 through 342may be performed in a different order than the order implied by theorder of these functional units in the image processing pipelineillustrated in FIG. 3, or may be performed by different functionalcomponents than those illustrated in FIG. 3. Moreover, the variouscomponents as described in FIG. 3 may be embodied in variouscombinations of hardware, firmware or software.

HOG Module

FIG. 4 is a block diagram illustrating a vision module 322 in the imagesignal processor 206, according to one embodiment. As described above,the vision module 322 performs various operations to facilitate computervision operations such as the generation of HOG data. FIG. 4 is a blockdiagram illustrating a histogram-of-oriented gradients (HOG) module 400included in the vision module 322 of the ISP 206, according to oneembodiment. Generally, the HOG module 400 processes images to create HOGdata for each image. An example of HOG data is a histogram-of-orientedgradients that is generated for an image based on identified gradientorientations within the image. The HOG data can be used in variouscomputer vision applications such as image classification, scenedetection, facial expression detection, human detection, objectdetection, scene classification, virtual reality, augmented reality, andtext classification.

As shown in FIG. 4, the HOG module 300 may include, among othercomponents, an input buffer circuit 401, a HOG engine circuit 403, anoutput buffer circuit 405, and configuration registers 407. The HOGmodule 300 may include circuits other than or in addition to thoseillustrated in FIG. 4 in other embodiments.

The input buffer circuit 401 receives image data of an image forprocessing by the HOG module 400 and stores the received image databefore it is processed by the HOG engine circuit 403, as describedbelow. In one embodiment, the image data received by the input buffercircuit 401 is either 8-bit image data format or 12-bit image dataformat. Generally, the image data received by the input buffer circuit401 includes only a certain component of a color image. For example, theimage data may be luma data that describes brightness information of theassociated image. The image data may alternatively be chroma data thatconveys color information of the associated image. Alternatively, theimage data may be raw image data. The image data may be received by theinput buffer circuit 401 from the sensor interface 302 or from otherimage sources such as directly from the image sensor 202 or other imagedata sources.

The HOG engine circuit 403 processes image data received from the inputbuffer circuit 401 to generate HOG data of the image data. In oneembodiment, HOG data is also referred to herein as gradient features ofthe image data. As mentioned above, a histogram-of-oriented gradients isan example of HOG data generated by the HOG engine circuit 403. Ahistogram of oriented gradients is a feature descriptor that describesthe gradients of the image data. Operation of the HOG engine circuit 403is described below with respect to FIG. 5.

The HOG engine circuit 403 outputs HOG data to the output buffer circuit405. The output buffer circuit 405 stores the HOG data until the HOGdata is ready for output according to the central control module 320.The output buffer circuit 405 may output HOG data in different bitformats. For example, the output buffer circuit 405 may output the HOGdata in an 8-bit format or a 16-bit format. The bit format of the HOGdata outputted by the output buffer circuit 405 may be different or thesame as the bit format of the image data received by the input buffercircuit 401. In one embodiment, the output buffer circuit 405 outputsthe HOG data directly to a convolution module (not shown). Theconvolution module may be included in the vision module 322 and performsfunctions such as edge map generation and/or image smoothing using theHOG data.

As shown in FIG. 4, the HOG module 400 also includes configurationregisters 407. The configuration registers 407 are configurableregisters that store configuration values transmitted by the centralcontrol module 320. The HOG module 400 sets operation parameters of thecomponents of the HOG module 400 illustrated in FIG. 4 based on theconfiguration values stored in the configuration registers 407. That is,the operation parameters of the input buffer circuit 401, the HOG enginecircuit 403, and the output buffer circuit 405 are set according to theconfiguration values stored in the configuration registers 407 by thecentral control module 320.

In one embodiment, each configuration register is associated with aspecific operation parameter of the HOG module 400 and the configurationvalue stored in each configuration register determines how theassociated operation parameter for the HOG module 400 is set. Forexample, whether the input buffer circuit 401 receives 8-bit or 12-bitimage data is based on a value stored in a configuration registerassociated with the bit format of the image data. The differentoperation parameters of the HOG module 400 are described below withrespect to FIG. 5.

FIG. 5 illustrates a detailed view of the HOG engine circuit 403according to one embodiment. The HOG engine circuit 403 includes anengine core 501. The engine core 501 performs an algorithm forgenerating HOG data for image data received from the input buffercircuit 401. An algorithm for generating HOG data is well known to thoseskilled in the art of image processing and is described in brief detailbelow.

To generate the HOG data for image data, the engine core 501 firstcomputes the gradients of the image data. To compute the gradients, theengine core 501 applies a mask to the image data in one or both of thehorizontal and vertical directions to determine the gradients of theimage data. For example, the engine core 501 may apply a 3×3 Sobel maskto the image data to identify the horizontal and vertical gradients ofthe image data within a cell defined by the mask. Other masks may alsobe used. The engine core 501 may optionally ignore any boundarygradients located at the boundaries of the image data.

The engine core 501 generates a histogram of gradient features based onthe computed gradients of the image data. Specifically, the engine core501 generates a histogram-of-oriented gradients for each cell defined bythe mask that is applied to the image data. A histogram-of-orientedgradients is an example of HOG data generated by the HOG module 400. Thehistogram-of-oriented gradients for each cell defined by the appliedmask describe the distribution of the orientation of gradients of theimage data within the cell. The HOG engine circuit 403 may include, forexample, 16 orientation bins in a histogram-of-oriented gradients whereeach orientation bin is associated with a range of orientations. Thenumber of orientation bins included in the histogram-of-orientatedgradients is configurable in one embodiment. The orientation bins areevenly spread over 0 to 180 degrees (“unsigned”) or 0 to 360 degrees(“signed”), depending on whether the gradients are “unsigned” or“signed.” Unsigned or signed HOG data that is not normalized by thenormalizer 505 described below is referred to as “non-normalized HOGdata” as shown in FIG. 6. FIG. 6 illustrates the different types of HOGdata generated by the HOG engine circuit 403 according to oneembodiment.

To create a histogram-of-oriented gradients for a cell, the engine core501 determines for the target pixel within the cell a weight for anorientation-based histogram bin based on the value of the gradientcalculated in the gradient computation. The engine core 501 increasesthe count of an orientation bin if the value of the gradient for thepixel is a value is within the range of orientations associated with theorientation bin.

In one embodiment, pixel contribution to an orientation bin is based ongradient magnitude. The engine core 501 determines the gradientmagnitude for each pixel based on the root of the squared sum of thepixel difference in the horizontal direction (dx) for the pixel and thepixel difference in the vertical direction (dy) for the pixel. Thus,rather than merely increasing the count of an orientation bin if thevalue of the gradient for the pixel is within the range of orientationsassociated with the orientation bin, the engine core 501 increases thecount of an orientation bin according to the magnitude of the gradient.The engine core 501 may also identify the angle for each pixel bycalculating the arc tangent of the ratio of the pixel difference in thehorizontal direction (dx) for the pixel and the pixel difference in thevertical direction (dy) for the pixel.

As shown in FIG. 5, the HOG engine circuit 403 also includes anormalizer 505 in one embodiment. The normalizer 505 optionallynormalizes histograms-of-oriented gradients generated by the HOG enginecircuit 403 to smooth the energy represented in the histograms. Thenormalizer 505 may use any normalization scheme to normalize thehistograms such as L2-norm, L2-hys, L1-norm, and L1-sqrt. However, othernormalization algorithms may be employed.

In one embodiment, the normalizer 505 is enabled or the type ofnormalization algorithm to be used is determined based on theconfiguration of the configuration registers 407. That is, theconfiguration registers 407 includes one or more configuration registersthat are associated with the normalizer 505. The normalizer 505 may beenabled to normalize HOG data according to the values set in theconfiguration registers associated with the normalizer 505.

As mentioned above, HOG data may be “signed” or “unsigned.” In oneembodiment, signed HOG data that is normalized by the normalizer 505 isreferred to as “signed HOG data” as shown in FIG. 6. Similarly, unsignedHOG data that is normalized by the normalizer 505 is referred to as“unsigned HOG data” as shown in FIG. 6. The use of “signed HOG data” and“unsigned HOG data” is based on the application and is typically used todifferentiate between similar features. For example, “signed HOG data”is used in an application to differentiate between a black-to-white 45degree edge and a white-to-black 45 degree edge. In contrast, if anapplication considers both the black-to-white 45 degree edge and thewhite-to-black 45 degree edge equivalent, the application uses the“unsigned HOG data.”

The engine core 501 may optionally calculate a sum of all theorientation bins from the normalized signed HOG data. The sum of all theorientation bins from the normalized signed HOG data is referred to as a“summed magnitude of HOG data” shown in FIG. 6. The summed magnitude ofnormalized HOG data is a single number that indicates how the regionwithin a cell is “textured” or whether it has edges with largemagnitudes. As mentioned, above, the value of an orientation-based binis increased when a particular orientation of a pixel is identified. The“summed magnitude of HOG data” is used by applications to differentiatebetween similar features. For example, the “summed magnitude of HOG datais used by applications to differentiate between similar features withdifferent amounts of texture such as a diagonal line in a whitebackground and a diagonal line in a background of grass.

To calculate the sum of the normalized signed HOG data, rather thanincreasing the value of an orientation-based histogram bin when aparticular orientation associated with the bin is identified, the enginecore 501 increases the value of an orientation bin according to themagnitude of the edge from the image data having the orientationassociated with the orientation bin. Thus, even if the image data hasthe same angle distribution across the different orientation bins, theorientation histogram generated for by the engine core 501 for aparticular orientation bin has larger values if the edges assigned tothe bin have a larger magnitude (e.g., a higher local contrast). Oncethe engine core 501 has identified the HOG data for received image data,the engine core 501 sums the values included in the orientation-bins togenerate the summed magnitude of HOG data for the image data.

Similar to the “summed magnitude of HOG data,” the engine core 501 mayoptionally calculate a sum of all the orientation bins from thenon-normalized signed HOG data. In one embodiment, the sum of all theorientation bins from the non-normalized signed HOG data is referred toas a “summed magnitude of non-normalized HOG data” as shown in FIG. 6.The engine core 501 calculates the “summed magnitude of non-normalizedHOG data” in a similar manner as the “summed magnitude of HOG data”described above except using the non-normalized HOG data rather than thenormalized HOG data. Similar to the “summed magnitude of non-normalizedHOG data,” the “summed magnitude of non-normalized HOG data” is used todistinguish between similar features with different textures.

The engine core 501 may also identify the total number of edges in theimage data that have a magnitude less than a threshold magnitude. Thethreshold magnitude may be established by the designer of the visionmodule 322. The engine core 501 calculates the magnitude of each edge inthe image data and increments a count responsive to the magnitude of anedge being less than the threshold magnitude. The total number of edgesin the image data that have a magnitude less than the thresholdmagnitude is referred to as “low magnitude HOG data” as shown in FIG. 6.In one embodiment, the “low magnitude HOG data” is used by anapplication to differentiate between similar features with differentamounts of local contrast.

Returning back to FIG. 5, in one embodiment the HOG engine circuit 403includes a patch mode processor 503. The patch mode processor 503 mayinclude similar functionality as the engine core 501 and is optionallyenabled to cause the HOG engine circuit 403 to operate in a patch mode.In contrast to the engine core 501, the patch mode processor 503 canprocess image data of an image that includes multiple image patchesduring the patch mode. The central control module 320 may provide theimage data of the image that includes multiple image patches to the HOGmodule 400.

FIG. 7 illustrates an image 700 that includes a plurality of imagepatches that are processed by the patch mode processor 503. Image 700includes a patch 701, a patch 703, a patch 705, and a patch 707. Thepatch mode processor 503 identifies the boundaries of the differentpatches and ignores the pixels located at the boundaries duringprocessing of image 700 in one embodiment. The patch mode processor 503may generate histograms-of-oriented gradients for the multiple imagepatches included in the image 700 as described above with respect to theengine core 501. That is, the patch mode processor 503 generates ahistogram-of-oriented gradients for patch 701, patch 703, patch 705, andpatch 707.

In one embodiment, each image patch included in the image receivedduring the patch mode may be extracted from a distinct image. Thus, theimage received during the patch mode may be composed of image patchesfrom different images. For example, FIGS. 8A to 8D illustrate distinctimages that each have an image patch included in the image 700 shown inFIG. 7. The different images shown in FIGS. 8A to 8D are considered thesource images for the image patches included in image 700.

Specifically, FIG. 8A illustrates in image of a star with image patch701 indicating an area of interest in the image of the star. FIG. 8Billustrate an image of a circle with image patch 703 indicating an areaof interest in the image of the circle. FIG. 8C illustrate an image of adiamond with image patch 705 indicating an area of interest in the imageof the diamond. Lastly, FIG. 8D illustrate an image of a rectangle withan image patch 707 indicating an area of interest in the image of therectangle. In one embodiment, the central control module 320 specifiesthe locations of the image patches within each image shown in FIGS. 8Ato 8D for processing during the patch mode. In one embodiment, theconfiguration registers 407 store the location of the image patcheswithin the source images.

In another embodiment, each image patch included in the image forprocessing during the patch mode may be extracted from a single sourceimage. When the image patches are extracted from a single source image,each image patch represents a different area of interest of the singlesource image. Each image patch may be specified by the central controlmodule 320. By operating in the patch mode, the HOG module 400 canefficiently process the different image patches in a single image ratherthan process the different images from which the different image patcheswere extracted or process the entire source image from which the imagepatches were extracted.

Returning back to FIG. 5, as described above, the HOG module 400includes configuration registers 407. The configuration values stored inthe configuration registers 407 are set by the central control module320. Operation parameters of the HOG module 400 are set based on theconfiguration values stored in the configuration registers 407. Anexample of an operation parameter is a bit format of the image datareceived by the HOG module 400. For example, whether the input buffercircuit 401 receives 8-bit or 12-bit image data is based on aconfiguration value stored in a configuration register associated withthe bit format of the image data. Another example of an operationparameter is the type of image represented by the image data. Forexample, whether the input buffer circuit 401 receives luma image dataor chroma image data is based on a value stored in a configurationregister associated with the bit format of the image data.

Another example of an operation parameter is a bit format of the HOGdata outputted by the HOG module 500. For example, whether the outputbuffer circuit 405 outputs HOG data in an 8-bit format or 12-bit formatis based on a value stored in a configuration register associated withthe bit format of the output buffer circuit 405.

With respect to the HOG engine circuit 403, as described above the HOGengine circuit 403 may output different types of gradient features shownin FIG. 6 which are considered operation parameters. The different typeof gradient features include signed HOG data, unsigned HOG data, summedmagnitude of HOG data, non-normalized HOG data, low magnitude HOG data,and summed magnitude of non-normalized HOG data. Each type of gradientfeature is enabled for output by the HOG engine circuit based on theconfiguration of the configuration registers 407. In one embodiment, atleast one of the different types of gradient features may be enabled foroutput. Furthermore, the number of orientation bins included in HOG datais configurable based on the configuration of the configurationregisters 407.

Lastly, the settings of the configuration registers 407 may alsodetermine whether the HOG engine circuit 403 operates in the patch modeand whether the HOG data is normalized as described above. Responsive tothe configuration registers associated with the patch mode andnormalization storing values to enable patch mode and normalizationfunctionality, the HOG engine circuit 403 enables the patch modeprocessor 503 and normalizer 505.

Example Process of Generating HOG Data

FIG. 9 illustrates a flowchart for generating histograms of gradientfeatures according to one embodiment. Note that in other embodiments,steps other than those shown in FIG. 9 may be performed.

In one embodiment, the central control module 320 generates andtransmits 901 configuration values to the configuration registers 407.The HOG module 400 sets 903 the operation parameters of the componentsof the HOG module 400 based on the received configuration values fromthe central control module 320. For example, the operation parameters ofthe input buffer 401 and/or the HOG engine circuit 403 are set based onthe configuration values of the configuration registers 407.

The HOG module 400 receives 905 an image for processing. The HOG module400 generates 905 histograms of different types of gradient features ofthe received image data based on the configuration of the configurationregisters 407. The different types of gradient features include signedHOG data, unsigned HOG data, summed magnitude of HOG data,non-normalized HOG data, low magnitude HOG data, and summed magnitude ofnon-normalized HOG data. Each type of gradient feature is enabled foroutput by the HOG engine circuit based on the configuration of theconfiguration registers 407. The HOG module 400 then outputs 909 thegenerated histograms of different types of gradient features as definedin the configuration of the configuration registers 407.

FIG. 10 illustrates a flowchart for processing image data during thepatch mode according to one embodiment. Note that in other embodiments,steps other than those shown in FIG. 10 may be performed.

In one embodiment, the HOG module 400 receives 1001 image data forprocessing that includes a plurality of image patches. The image patchesmay be from different source images or from a single source image.

The HOG module 400 identifies 1003 boundaries of the plurality ofpatches included in the image in order to ignore the pixels located atthe boundaries. The HOG module 400 generates 1005 a histogram ofgradient features for each of the plurality of patches included in theimage based on the identified boundaries. The HOG module 400 outputs1007 the generated histograms of gradient features for the plurality ofpatches included in the image.

The foregoing description, for purpose of explanation, has beendescribed with reference to specific embodiments. However, theillustrative discussions above are not intended to be exhaustive or tolimit the invention to the precise forms disclosed. Many modificationsand variations are possible in view of the above teachings. Theembodiments were chosen and described in order to best explain theprinciples of the invention and its practical applications, to therebyenable others skilled in the art to best use the invention and variousdescribed embodiments with various modifications as are suited to theparticular use contemplated.

Finally, the language used in the specification has been principallyselected for readability and instructional purposes, and it may not havebeen selected to delineate or circumscribe the inventive subject matter.It is therefore intended that the scope of the invention be limited notby this detailed description, but rather by any claims that issue on anapplication based hereon. Accordingly, the disclosure of the embodimentsof the invention is intended to be illustrative, but not limiting, ofthe scope of the invention, which is set forth in the following claims.

What is claimed is:
 1. A configurable histogram-of-oriented gradients(HOG) module comprising: a plurality of registers configured to storeconfiguration values including a first configuration value; a HOG enginecircuit coupled to the plurality of registers to receive the firstconfiguration value, the HOG engine circuit configured to receive imagedata and generate histograms of different types of HOG features of theimage data based on at least the first configuration value received fromthe plurality of registers, the different type of HOG features of theimage data including a total number of edges in the image data with amagnitude less than a threshold; and an output interface circuit coupledto the HOG engine circuit, the output interface circuit configured tooutput the histograms of different types of HOG features generated bythe HOG engine circuit to a computer processing unit coupled to the HOGmodule.
 2. The configurable HOG module of claim 1, further comprising:an input interface circuit configured to receive the image data andprovide the image data to the HOG engine circuit.
 3. The configurableHOG module of claim 1, wherein the HOG engine circuit comprises: anormalizer configured to normalize at least one of the different typesof HOG features according to a second configuration value stored in theplurality of registers.
 4. The configurable HOG module of claim 3,wherein the different types of HOG features of the image data includenon-normalized HOG features and the HOG engine circuit generatesnon-normalized HOG features of the image data based on a thirdconfiguration value stored in the plurality of registers.
 5. Theconfigurable HOG module of claim 3, wherein the different types of HOGfeatures of the image data include a summation of magnitudes of edgesincluded in the image data and the HOG engine circuit is configured tocalculate the summation by (i) calculating a magnitude of each edgeincluded in the image data and (ii) summing the magnitudes of the edgesbased on a third configuration value stored in the plurality ofregisters.
 6. The configurable HOG module of claim 5, wherein thesummation is based on normalized HOG features of the image data.
 7. Theconfigurable HOG module of claim 5, wherein the summation is based onnon-normalized HOG features of the image data.
 8. The configurable HOGmodule of claim 1, wherein the HOG engine circuit is configured todetermine the total number of edges by: calculating a magnitude of eachedge in the image data; and determining the total number of edges havingmagnitudes less the threshold.
 9. The configurable HOG module of claim1, further comprising a patch mode processor coupled to the HOG enginecircuit, the patch mode processor configured to generate a histogram ofgradient features for each of a plurality of patches included in theimage data.
 10. The configurable HOG module of claim 9, wherein each ofthe plurality of image patches is extracted from different sourceimages.
 11. The configurable HOG module of claim 9, wherein each of theplurality of image patches is extracted from a single source image. 12.A method, comprising: setting first operation parameters of componentsof a histogram-of-oriented gradients (HOG) module based on a firstconfiguration value stored in a plurality of configuration registersincluded in the HOG module; receiving image data for processing by theHOG module; generating, by the HOG module, histograms of different typesof HOG features of the received image data based on at least the firstoperation parameters set according to the first configuration valuestored in the plurality of configuration registers the different type ofHOG features of the received image data including a total number ofedges in the received image data with a magnitude less than a threshold;and outputting the generated histograms of different types of HOGfeatures to a computer processing unit coupled to the HOG module. 13.The method of claim 12, further comprising: receiving the firstconfiguration value of the plurality of configuration registers from thecomputer processing unit coupled to the HOG module.
 14. The method ofclaim 12, further comprising: normalizing at least one of the differenttypes of HOG features according to a second configuration value storedin the plurality of configuration registers.
 15. The method of claim 14,wherein the different types of HOG features of the image data includenon-normalized HOG features and the method further comprising:generating non-normalized HOG features of the image data based on athird configuration value stored in the plurality of registers.
 16. Themethod of claim 14, wherein the different types of HOG features of theimage data include a summation of magnitudes of edges included in theimage data and the method further comprising: calculating a magnitude ofeach edge included in the image data; and summing the magnitudes of theedges based on a third configuration value stored in the plurality ofconfiguration registers.
 17. The method of claim 16, wherein thesummation is based on either normalized HOG features of the image dataor non-normalized HOG features of the image data.
 18. The method ofclaim 12 further comprising: calculating a magnitude of each edge in theimage data; and determining the total number of edges having magnitudesless the threshold.